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Abstract 

Technology has been part of the language learning process, both inside and outside the classroom, for decades, 
helping to bridge the gap between different contexts of learning. At the same time, the concept of lifelong-learning 
has widened our understanding of what learning is, what different forms it can take, and of the importance of 
considering the learning process as one that will continue once formal education is over and the learner has entered 
the world of employment. 

Learner autonomy has also taken a central role, and has brought with it a shift in the role of the teacher in the 
language learning process. Self-assessment is one aspect of learner autonomy, and is crucial in enabling learners to 
set themselves clear objectives and thus take responsibility for the leaning process itself. 

In addition, the importance of including the development of intercultural competence in the language classroom has 
been advocated by many linguists and educationalists (Kramsch, 1986; Byram et al., 2001; Byram, 2008; Corbett, 
2003; Sercu, 2005). 

It is against this background that the European-funded project CEFcult (2009-2011) was developed. Its main 
outcome is the production of an online environment for the collaborative assessment of oral skills and intercultural 
competence in the foreign language. The target groups are language teachers, in-company trainers, those about to 
start or return to work, and undergraduates and graduates who are preparing themselves for first-time employment. 
The tool places side by side the widely known and standardised descriptors of the Common European Framework of 
Reference (CEFR) for spoken production and interaction, and the descriptors for intercultural competence in a 
professional setting, developed within the Intercultural Competence Assessment (INCA) project. 

CEFcult provides a set of authentic communication scenarios, each containing various tasks through which learners 
can assess their skills for intercultural professional purposes in different European languages. Although the tool can 
also be used autonomously, in a classroom setting, teachers can set the learners suitable scenarios, get them to 
perform the speaking tasks and record the oral communication activities. After uploading the samples onto the online 
tool, they can be assessed by the teacher, by other experts (including teachers in other countries), or by other learners. 
The tool can also be used for self-assessment. By encouraging self- and peer-assessment, CEFcult can positively 
engage learners in a process of reflexive learning that will lead to greater autonomy. However, it also enables the 
teacher to be an active part of this same process. 

After a brief introduction to the theoretical background and pedagogical choices underpinning the project, we will 
illustrate the pilots earned out at the universities of Dresden and Bologna, and discuss some of the implications for 
learners and teachers. 
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1. Introduction 

CEFcult (2009-201 1) is a European project funded by the EU Lifelong Learning Programme. Building on 
the WebCEF project (2006-2009), its main objective was the development of an online assessment 
environment for oral language skills and intercultural communicative competence in professional 
communication. In this way, CEFcult brings together the needs of businesses and of educational 
institutions in Europe. 

2. The online environment 

The online environment contains first of all a number of scenarios (tasks) designed to elicit specific 
linguistic performance in an intercultural context. The user can complete the chosen scenarios and record 
his or her oral production directly onto the online tool. These video clips can then be used for self-, peer- 
or expert assessment, using a grid containing the CEF descriptors for oral production, as well as the INCA 
descriptors for intercultural competence. As well as the scenarios and assessment tool, the environment 
also contains training materials. 

3. The scales 

The scales used in CEFcult are those included in the Common European Framework of Reference for 
Languages (CEFR). The main scales used as those for linguistic competence (oral production and 
interaction): range, accuracy, fluency, coherence and interaction. However, some scenarios are designed 
to be assessed with the sociolinguistic competence and the pragmatic competence scales, which are more 
strongly related to intercultural communicative competence. 

3.1. Intercultural Competence Assessment (INCA) 

The INCA scales (www.incaproject.org) assess six different elements of competence: respect for 
otherness; tolerance of ambiguity; knowledge discovery; empathy; behavioural flexibility; communicative 
awareness. These elements are defined taking into account three factors: motivation, knowledge/skills, 
and behaviour. 

Although space constraints make it impossible to analyse all six elements and their descriptors, as an 
example we shall present “Respect for Otherness”. This, according to the INCA theory document 
(available from www.incaproiect.org ), is “manifested in curiosity and openness, readiness to suspend 
belief about the ‘naturalness’ of one’s own culture”. It can be observed in the individual’s willingness “to 
suspend usual assumptions and seek out new knowledge from others in a relationship of equality and, as a 
consequence, to question their own existing assumptions” (motivation), in their “knowledge of their own 
assumptions and those of their society, and the ability to critically assess the logic of a system of beliefs, 
values and behaviours” (knowledge/skills), and in the interest shown “in the other’s experience and in the 
daily experience of a range of social groups within a society, and not only that represented in the 
dominant culture” (behaviour). The table below shows the descriptors for the three levels of competence. 


Level 

Openness: respect for otherness 

Basic 

Is not always aware of difference and, when it is recognised, may 
not be able to defer evaluative judgement as good or bad. Where it 
is fully appreciated, adopts a tolerant stance and tries to adapt to 
low -involving demands of the foreign culture 

Intermediate 

Accepts the other's values, norms and behaviours in everyday 
situations as neither good nor bad, provided that basic assumptions 
of his own culture have not been violated. Is motivated to put 
others at ease and avoid giving offence. 

Full 

Out of respect for diversity in value systems, applies critical 
knowledge of such systems to ensure equal treatment of people in 
the workplace. Is able to cope tactfully with the ethical problems 
raised by personally unacceptable features of otherness. 


Table 1 - INCA descriptors for “Respect for Otherness”. 
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4. Scenario: Working in a Multinational Team in Europe 

As mentioned above, in order to assess (or have assessed) their intercultural competence, learners have to 
complete tasks on the CEFcult platform. These tasks are grouped in larger “scenarios”, in the sense of 
hypothetical situations, specifically created to elicit the learner’s oral production. The scenario can consist 
of trigger texts and / or videos on relevant topics. 

Within the CEFcult project, a number of scenarios were developed, including "Taking part in an 
online screening interview for a career in an international organization" with tasks such as presenting 
oneself, discussing one’s suitability for the desired post or describing previous experiences. Another 
example is the scenario "Participating in an international summer course for foreign language teachers" 
which aims, among other things, at raising awareness of some features of one’s communication style. 

The scenario presented here is called: "Working in a multinational team in Europe", and was designed 
for the individual user. Some tasks are based on those created by the INCA project team 
( http://www.incaproject.org ). The scenario is made up of six tasks, each testing one element of 
intercultural competence as defined in INCA. The setting is defined in the introduction: 

You have been selected to take part in a two-year long work project with 
European partners. The project language will be English, used as a 
lingua franca. This will involve your spending 3-4 weeks in another 
European country, but also some meetings abroad, written and spoken 
online collaboration, and hosting the members of the team in your own 
country. 

The six tasks concern aspects such as finding information before travelling abroad, attending 
meetings, socialising with people from other countries, hosting colleagues, chairing meetings etc. The 
learner is typically asked to imagine a hypothetical situation and react to it. 

The CEFcult tool, which is freely available online and extremely user-friendly, requires the learner to 
record him or herself directly onto the tool using a webcam. The responses are saved and can then be 
subjected to self-, peer- or expert assessment, by using the grids containing the CEF and INCA scales. It 
is the user him or herself who chooses who has access to the recorded samples. (For a detailed description 
on how to use the CEFcult tool please refer to the user manual available from the website: 
www.cefcult.eu ) 

5. Piloting the assessment of Task 5, relating to “Respect for Otherness” 

As an example of the piloting recordings, the video response of speaker J. for task 5 was presented to the 
conference audience, with the aim of raising their awareness of some of the complexities of this kind of 
assessment. J. was an exchange student in Bologna at the time of the piloting. Task 5 is an oral production 
activity, which intends to elicit the speaker's competence for the element “Respect for Otherness”: 

One of the behaviours which many colleagues find frustrating concerns 
time-keeping: some colleagues tend to arrive late to the meeting while 
others are extremely punctual. Some start leaving the meeting before it 
has ended, while others always stay until the end. The project coordinator 
has asked you to chair the next meeting. He/she has also asked you to 
moderate in order to try to make the meeting as successful as possible, 
and make it less frustrating for those who feel less at ease. Now imagine 
you are opening the meeting, and explaining to the colleagues what you 
could all do to deal successfully with the issue of time-keeping. When you 
are ready, record yourself while you give this initial presentation . 

The length of the sample is 2 minutes 36 seconds. The audience was asked to assess this speaker's 
competence using the INCA descriptors for the specified element. In the lively discussion that followed 
most people agreed that he could be granted no more than a basic level of competence for this task. The 
arguments were that J. resorted to common stereotypes about the unpunctuality of "southerners", that he 
seemed to impose his own values on others by taking it for granted that his own value system (regarding 
punctuality) was better than that of others. 

However, it was clear that this task posed some problems: for example, it raised the issue of whether 
something as complex as intercultural competence can be assessed on a hypothetical situation rather than 
on authentic behaviour. 

6. Conclusions 

As a result of the piloting sessions, a number of conclusions were reached. 
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• In general, it was agreed that the CEFcult tool seemed particularly useful for raising awareness 
of Intercultural Communicative Competence. However, it also became clear that assessing ICC 
was an extremely complex procedure and should therefore be used with extreme care, if at all, 
in high-stakes assessment. 

• The internal validity of the tasks was even more problematic for ICC than for language 
assessment: some tasks proved to be unsuitable as they did not elicit the speaker’s knowledge/ 
skills/ attitudes for the desired INCA element, and needed to be reformulated. 

• The usefulness of self-assessment is not necessarily obvious to all students. In addition, it is 
important that learners can compare their self-assessment with other peer- or expert 
assessments of the same samples. 

• Some learners considered that culture-general scenarios were not particularly useful, and called 
for culture- specific situations (for example, having to work in Madrid, doing business with a 
French partner, etc.). 

• Finally, as mentioned above, a major concern amongst the project partners was the artificial 
setting of the scenario, in which the learner was asked to imagine talking to or interacting with 
a partner, but was in fact recording a monologue. This makes it impossible to react to what is 
being said, which is usually the case in real life. It is in fact arguable whether the assessment of 
intercultural competence can be limited to one of the speakers in a dialogue. 

• Another issue raised was whether intercultural competence may be impeded by lack of 
sufficient linguistic skills. In other words, learners with a low level of proficiency may not be 
capable of showing their intercultural competence to the full. 

• Linked to this last point is the fact that the INCA framework was not written to assess 
Intercultural Communicative Competence in the foreign language. A future development may 
be to develop scales for ICC within the CEF. 
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